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ABSTRACT 
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time. Analytic procedures fcr evaluating prograiT:s with multiple 
oDjectives are discussed. Fart 3 discussas cost-benefit 
considei"ation.? m school-basec and company-based public vocational 
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located at New United Motors Manufacturing, Ire, ano considers who 
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This paper seeks to clarify the public purposes of postsccondary vocational education 
C 'VE), and proposes methods by which these purposes may be more effectively implemented. 
The paper consists of three parts. Part I reviews the public purposes or PVE. These are 
postulated: to increase individuals' satisfaction at work, by facilitating appropriate choices of 
occupation; to increase the employment of under-represented groups in occupations to which 
they have been denied access; and to increase the economy's total output of goods and services. 
PVE can accomplish this last purpose by increasing the supply of labor or by increasing the 
productivity of labor and capital. 

Defining these purposes is more straightforward than measuring the degree to which they 
are achieved. The most commonly used outcome measures arc placement rates and wage gains of 
individuals who have spent time in PVE. However, evidence on employment and earnings of 
individuals docs not necessarily imply whether PVE is achieving its public purposes. It is 
possible that individuals could experience more economic success after PVE, even while the 
program as a whole contributes nothing to total economic output, access, or job satisfaction. This 
seeming paradox is discussed in the first part of the paper. The conclusion is that labor market 
outcomes for PVE graduates are not sufficient forjudging whether the program accomplishes its 
public purposes. However, economists argue that positive labor market outcomes arc a necessary 
indicator of program effectiveness; th?t is, no program can be effective if participants do not 
obtain employment. By the same reasoning, no progiam can be effective if participants leam 
nothing from it. The implication is that measures of 'earning and labor market outcomes should 
both be included. 

Part n discusses how to handle multiple measures of pxrrformance. Various tradeoffs may 
exist. For txamplc, in selecting participants for the program there is a tradeoff between 
improving access of under-represented groups ar»d increasing total economic output if members 
of some under-represented groups on average require more costly instruction or other services in 
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order to succeed in Ac program. For any individual participant in PVE, the simple fact that time 
is limited mcams that some sicills and knowledge can be taught only at the expense of other 
subject matter. Given tradeoffs in producing the various desired outcomes of PVE, it is 
necessary to consider multiple outcomes simultaneously in evaluating PVE programs. Analytical 
methods for doing this arc described in the second part of the paper. Measuring efficiency in 
attainment of measured objectives is a more tractable problem than deciding exactly what 
objectives to measure in the first place. 

The third part discusses implementation. Various public agencies, private firms, and 
nonprofit organizations can provide PVE. Schools and employers have done most of it in the 
U.S., though community-based organizations have played a major part in training sponsored by 
CETA and JTPA. The degree to which particular programs in schools or firms produce benefits 
for taxpayers should be reflected in differential rates of governmental subsidy. A strategy for 
implementing performance-based policy toward PVE is briefly described. 

I. The Public Purposes of Postsecondary Vocational Education 

Three major purposes of PVE will be discussed here. These are: improving individuals' 
satisfaction in their work, increasing access of under-represented groups to certain occupations, 
and augmenting the economy's total output of goods and services. This list excludes preparation 
for the responsibilities of citizenship or parenthood, and enhancing the value of leisure time 
(except as leisure is enhanced by more goods and services). Excluding these goals acknowledges 
that the purposes of vocational education are narrower than the purposes of education in general. 

What makes the three stated purposes public rather than private is that they refer to 
aggregate outcomes. It serves no public purpose if one person's job -satisfaction or productive 
output is increased at the cost of someone ease's being decreased by the same aniount — unless 
the person being helped is a member of a more deserving under-represented group. It is the 
aggregate nature of these public purposes that makes evidence about individual outcomes, by 
themselves, inconclusive. This will be explained further below. 

"Public" here docs not mean governmental. Private individua!* who participate in PVE, and 
their employers, are part of the public. Benefits to employees and employers arc therefore 
considered public benefits. The net benefit of a program to the public may be positive even if the 
taxes paid on additional earnings or output do not exceed the program's cost to government. 
WheUier government or taxpayers receive net benefits from PVE is an important question, and is 
considered separately in Part lH, 



Increase Non-Pecuniary Rewards from Work 

This is listed first here because it is so often ignored (for a notable exception, sec 
Schumacher, 1979). If PVE has the effect of making people more satisfied with their work than 
they would otherwise be, this is a real contribution. Sinc« non-pccuniaiy rewards from work are 
not included in conventional measures of national income or product, they have to be accounted 
for separately. 

This is diiiicult, because answers to questions about job satisfaction cannot be compared 
between individuals, unlike questions about income or hours of work. Different individuals 
presumably mean the same thing when they refer to a dollar of pay or a week of work. But two 
individuals in identical jobs could express different levels of satisfaction, depending on their 
tastes, endowments, and expccutions. Recognizing the multidimensionality of job satisfaction 
docs not makr individual assessments more comparable, and it complicates the problem: for 
instance, one person might like the physical surroundings and social contacts with co-workers on 
a particular job, but dislike the lack (or excess amount) of challenge, etc.; another person might 
have the opposite response. Economists have made some attempts to attach monetary values to 
non-pecuniary characteristics of jobs. They have relied on Adam Smith's hypothesis that, in a 
competitive labor market, disagreeable jobs would have to pay higher wages in order to attract 
people. However, these empirical efforts have yielded only partial success (Lucas, 1977; Smith, 
1979; Stem, 1979; Duncan and Holmlund, 1983). Non-monetaiy aspects of work have not yet 
yielded to researchers* efforts to apply a monetary meter stick. Accordingly, non-monetary 
rewarxis from work are customarily ignored in evaluations of PVE and other employment-related 
programs, even though these would be inversely correlated with income (among equally able 
ii.dividuals) if Adam Smith's hypothesis is true. 

Nevertheless, public policy toward vocational education and job training in this country 
staunchly preserves the right of individuals to choose occupations, in large part because 
individual satisfaction at work is so important. It would be un-American to propose a policy of 
assigning individuals to jobs without 'iking their preferences into account. Choice leads to 
greater non-pecuniary satisfaction, through the individual "pursuit of happiness" at work. 
Indeed, a whole profession of career counselors, and the research discipline of vocational 
psychology, have arisen to help individuals choose occupations in which they will be most 
satisfied. We are consumers of v ork, as David Riesman one remarked, and the principle of 
consumer sovereignty applies. Enabling individuals to enter occupations where they expect to be 
happy is certainly one purpose of PVE. 



Increase Access of Under -Represented Groups to Certain Occupations 

Choice of occupations is considered such an important right in this country that public 
policy has tried to extend it to those who have lacked it The federal and state governments have 
enacted laws prohibiting discrimination in cmplsjyment on grounds of race, sex, age, and other 
criteria. Employment and training programs have been created to help disadvanuged groups 
obtain desirable jobs. Although this was not among the originally stated purposes of federally 
supported vocational education, the federal law authorizing expenditures for vocational education 
has been amended m recent decades to include stipulations and set-asides for improving access of 
disadvantaged and handicapped students, and for reduction of sex-stereotyping. 

Like the other public purposes of PVE, the objective is defined in terms of aggregates. For 
example, increasing the access of women to jobs as automotive mechanics means increasing the 
proportion of all auto mechanics who are women. If one woman is placed as an auto mechanic at 
the expense of another woman, the public purpose is not ser/ed. If the pool of applicants for 
auto mechanics' jobs is predominantly male, it would be reasonably safe to assume that placing 
one woman will not displace another woman from that profession. In this instance, simple 
placement data would be reasonably conclusive in judging whether a PVE program was 
promoting sex equity. 

However, in occupations where there is an ongoing history of discrimination, employers 
may still relegate women to the back of the job-applicant queue, along with racial minorities, 
linguistic minorities, and other under-represented groups. It would be quite possible, therefore, 
that a female PVE graduate who got a job in one of these occupations would be displacing a male 
who belonged to another under-rcprescntr^ group. There would be a gain in sex equity, but 
perhaps no gain in overall equity. To be certain that PVE contributes to overall equity, it would 
be necessary \o keep track of the total proportions of people employed in the target occupations 
who belong tc •'ndcr-reprcsenied groups, and to sec how those proportions change as a result of 
placements from PVE programs. 

Increase Total Output of Goods arM Services 

The market value of goods and scrvic^:s producexj by the ecowmy in a given period of time 
is the Gross National Product (GNP). Modifications of the GNP concept have been proposed, to 
include non-market (household) production and to account for the costs of congestion and 
pollution associated with economic growth (Nordhaus and Tobin, 1972), but GNP is still the 
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conventional measure of total economic output For economists, the most obvious criterion for 
judging PVB would be whether its contribution to total output exceeded its cost 

PVE can increase total output in two distinct ways. It can increase the supply of labor, or it 
can increase the amount of output prcKluced by given amounts of labor and capital, i.e., it can 
increase factor productivit>'. 

Statements about the aggregate effects of PVE necessarily imply some sort of 
macrocconomic model, either implicit or explicit There is a virtue in being explicit, because the 
underlying assumptions then become clear. There is also a risk, because macrocconomic theory 
is driven by unresolved controversies, and no model will command universal assent (especially 
in an election year). Despite the risk, it is worth consir'cring a simple macrocconomic model. 
This is depicted in Figure 1, which is a diagram that appears in contemporary textbooks on 
macroeconomics. Its purpose is didactic; it does not include the complexities and dynamics 
necessary in a model that would be used for empirical forccasdng. Nevertheless, it serves the 
purpose of this paper. 



Figure 1 

Aggregate Demand and Short-Run Aggregate Supply 
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Figure 1 shows two curves crossing. One is an aggregate demand (AD) curve, and the 
other is aggregate supply (AS). The point where they intersect is where the price level and the 



volume of economic activity arc in equilibrium, with values p* and q*. The aggregate demand 
and supply curves themselves each represent a locus of equilibrium points. The aggregate 
dcnuind curve shows combinations of price level and GNP that equilibrate investment, exports, 
and financial markets. Given a certain nominal supply of money and a certain level of 
govenmient taxation and expenditures, a lower price level would entail less demand for money, 
which would be offset (through adjustments in interest rates) by higher levels of investment and 
net exports, thus higher GNP. That is why the aggregrate demand curve has a negative slope. 

The aggregate supply curve has a positive slope in the short run. This reflects an 
assumption that prices change faster than wage levels. Lower prices therefore imply higher real 
wages for some period of time. Employers respond to higher real wages by employing less 
labor, thus reducing total output. In the long run, however, lower prices would lead to lower 
money wages, as unemployment retarded wage growth. The long run aggregate supply curve is 
therefore a vertical line at the full-employment level of GNP. This is shown in Figure 2, where 
q** denotes full-employment GNP. (The price level corresponding to q** is determined by 
monetary and fiscal policies, which shift the aggregate demand curve.) 

Figure 2 

Aggregate Demand and Long-Run Aggregate Supply 




The effect of PVTE on GNP would occur by shifting the long-run aggregate supply curve to 
the right. This would happen through the interaction cf employers and employees in the labor 
market, a simple model of which is shown in Figure 3. Here DL denotes demand for labor. 
Given the quantity of productive capital and the parameters of the production function, the 
demand for labor is inversely related to the real wage. SL denotes supply of labor. Given such 
things as the age composition of the population and the perceived non-pecuniaiy rewards from 
paid work compared to other alternatives, the supply of labor is positively related to the real 
wage. The maiket-clearing wage level, w* in Figure 3, is where the supply and demand curves 
intersect However, the process of matching people to jobs takes time, so there are always some 
job vacancies and some unemployed people. The dashed line in Figure 3 shows what quantity cf 
labor would actually be employed at any given wage level. The horizontal distance between the 
dashed line and the demand curve is the number of vacant jobs; the distance between the dashed 
line and the supply curve is the number of unemployed people. At the market-clearing wage 
level, the numbers of vacant jobs and unemployed people are the same: L* - E* in Figure 3, 
where L denotes the amount of labor supplied or demanded, and E denotes actual employment. 

Figure 3 
The Labor Market 

Real 
wage 




Nurr.bers of jobs or 
people employed (FT^) 



If wages were below the nuricct-clcaring level, the number of job vacancies (the horizonta) 
distance between DL and the dashed line) would exceed the number of unemployed people (the 
horizontal distance between SL and the dashed line). This rarely occurs, if ever. Instead, the 
number of unemployed people almost always exceeds the number of job vacancies. Abraham 
(1983) has estimated that, in the United Sutcs> the number of unemployed people exceeds the 
number of job vacancies when the unemployment ratr is higher than three or four percent, which 
it almost always is. This implies that actual wages tend to remain above the markci-clcaring level. 
Econon-ists have been seeking to explain this for about fifty years. Current theories postulate the 
existence of an "efficiency wage," which is kept above the market-clearing level because 
employers want to induce skilled workers to sUy on the job, or because the threat of 
unemployment keeps employees working hard, or because norms of fairness in workplaces 
oblige employers to overpay low-paid woikcrs in order to be able to give high-paid workers their 
market^learing wage (Akerlof and YeUen, 1988). 

Even though labor markets do not clear in the usual sense, Figure 3 is useful for suggesting 
how PVE or other training programs can increase total output Fl-sl expanding PVE would 
make it cheaper and easier for some individuals to learn skills related to work they want to do. At 
any given wage level, therefore, the number of people available for work would increase. Some 
of the new supply of workers might be people who would otherwise have engaged in unpaid 
work at home or elsewhere, others might have been retired from, or not yet in, the labor force, 
and some might have been working part-time but now seek employment in an occupation for 
which PVE would prepare them to wo»k full-time. The result would be to shift the SL curve to 
the right This would lead to a higher level of employment and output, with a lower real wage. 

Expanding PVE could also increase total output by making the existing labor force more 
productive, even if t*ic number of workers available for work at a given wage level did not 
increase. In the framework devised by economists to account for the various causes of economic 
growth, education contributes directly to improvement in the "quality of labor" and indirectly 
increases ^'total factor productivity" (Denison, 1962; Jorgenson. 1984). The "labor quality" effect 
consists of education enabling people to perform more highly paid jobs. The effect on "total 
factor productivity" occurs when more highly educated workers, using a given capital stock, are 
able to use known technology more efficiently. This distinction is somewhat arbitrary. Growth- 
accounting research offers a plausit e procedure to account for the sources of growth, but not an 
empirical demonstration that education (or anything else) really causes growth. In particular, the 
measured contribution of education to labor quality depends on average differences in earnings 
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between individuals with different amounts of schooling, and these earnings differences could 
arise from the "screening** effect of schooling (Winkler, 1987) rather than from any real effect of 
schooling on productivity. Attempts to test directly whether the aggregate level of schooling is 
associated with total output over time have produced mixed results (Walters and Rubinson, 
1983). What direct evidence we have that education in fact does increase productivity comes 
from micro-level studies of small scale farming (Jamison and Lau, 1982; Wozniak, 1987), where 
the productivity of the enterprise can be directly related to the educational attainment of the 
operator. These studies have demonstrated that general schooling increases productivity. There is 
not yet any published evidence on whether PVE, in particular, increases productivity. 

If PVE does incxasc productivity, it causes the demand for labor to shift to the right, as 
employers want to employ more people at any given wage level. With a given supply of labor, 
the shift in demand for labor would increase total employment and output, and raise real wages. 

The two output-increasing effects of PVE — enlarging the supply of labor and enhancing 
factor productivity — might both occur at the same time. The effects on output and employment 
would be unambiguously positive. The real wage might increase or decrease, depending on the 
relative sizes cf the shifts in labor supply and demand, and on the relative steepness of the two 
curves in Figure 3. 

The process of increasing employment and output in response to PVE entails some 
redeployment of capital. When PVE increases the supply of labor, the reduction in real wage 
induces employers to substitute labor for capital at the same time that they expand output The 
expansion of output requires additional capital, so the net effect of an increase in labor supply 
might be to increase or decrease the amount of productive capital used. Similarly, an increase in 
labor productivity induces substitution of labor for capital, but also icads to expansion of output 
which entails use of more capital. 

If the net effect of PVE is to increase the amount of capital used, the additional capital must 
come either from abroad or from increased domestic saving. Capital would be attracted from 
abroad in response to higher returns on productive investment in this country. The same higher 
returns might also induce a larger amount of saving by domestic households, firms, and 
governments, although empirical studies of how interest rates affect aggregate saving have 
yielded mixed results. 



Can PVE Increase Employment by Filling Job Vacancies Faster? 

As shown in Figure 3, unemployed people are looking for work at the same time that 
employers are looking for people to fill vacant jobs. The process of matching people to jobs takes 
some tL-nc. This gives rise to "frictional" unemployment, which may be defined as the amount by 
which unemployment would be reduced if all existing job vacancies were filled. It seems 
obvious that speeding up the process of matching people to jobs would reduce frictional 
unemployment and therefore reduce total unemployment. If unemployed people lack the skills 
required for the jobs that arc vacant, and if this is an important reason why it takes time to fill 
those jobs, then it also seems obvious that rrining programs such as JTPA or PVE can reduce 
unemployment 

Some support for this conclusion comes from evidence compiled by Mcdoff (1982). He 
found that the job-matching process apparently slowed down in the U.S. labor market during the 
1970s. Specifically, at any given level of unemployment, the number of job vacancies — as 
indicated by the rate of help- wanted advertising — was larger during the 1970s than in the two 
preceding decades. (Equivalently, the amount of unemployment was greater at any given level of 
job vacancies.) Furthermore, this trend coincided with a large increase in the number of reports 
of skill shortages published in a set of 71 business and industry periodicals. Medoff concluded 
"with caution, that a significant fraction of the recent growth in labor market imbalance has to do 
with a worsening match between the skills possessed by labor force members and the skills 
required in jobs'* (p. 6). MedofTs evidence of more apparent job vacancies at any given level of 
unemployment, or vice versa, implies, in terms of Figure 3, that the distance between the dashed 
line and the solid supp'y and demand curves increased during the 1970s. 

Would total employment increase if the dashed line in Figure 3 could be shifted up closer to 
the supply and demand curves? If the supply and demand curves did not move, employment 
certainly would increase when the dashed line moved to the right The question is whether the 
supply and demand curves themselves would move. In particular, the question pertains to the 
demand curve, since employment is almost always limited by demand, not supply (unemployed 
people usually outnumber job vacancies). Would increasing the proponion of jobs offered 
(demand curve) that are actually filled (dashed line) have the effect of reducing the number of 
jobs offered at any given wage? If employers become aware that the chances of filling positions 
within a given period of dme have increased, because qualified people arc more readily available 
(thanks to PVE, JTPA, or some other program), will they advertise fewer positions? This seems 
likely, since there are costs of advertising and screening applicants. Employers are interested in 
achieving their employment targets at minimum cost They do not want to have unfilled 
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positions, but they also do not want to be swamped with applicants. Theirfors, as job vacancies 
are filled more quickly, fewer iobs would be offered. In Figur; 3, reducing the distance between 
the dashed line and the demand curve would induce a leftward shift of the demand curve ittelf. 
The gain in total employment, if any, would probably be very small. It would be very useful if 
empirical research could shed some light on this. In the meantime, it seems unwise to base public 
policy on the dubious assumption that filling job vacancies faster will increase total en^loyment 

The assumption that PVE can reduce unemployment by filling job vacancies is not 
necessary to the rationale for PVE's existence. Even if unemployment were completely 
eliminatr^, the public purposes of PVB would remain, including the purpose of increasing total 
output In terms of Figure 3, even if the dashed line disappeared so that the labor market cleared 
at wage w* and employnient E* «= L*, the purpose of PVE still would be to shift the demand and 
supply curves to the right, especially the demand curve. Enhancing productivity is a prime 
purpose of PVE even when the economy is at full employment 

Why Evidence on Labor Market Experience of PVE Graduates is Inconclusive 

Unfortunately, data on how many PVE graduates get jobs cannot resolve the question 
whether PVE increases total employment and output The reason is simply that some or all of the 
jobs obtained by PVE graduates would have been filled by other job-seekers if the PVE program 
did not exist The number of job-seekers displaced shoukl be subtracted from the number of PVE 
graduates who are placed in jobs, to give a net placement rate. Only if the net placement rate is 
positive would it be correct to say that PVE increased total employment and output by reducing 
job vacancies. In practice, net placement rates have not been collected, though it might be 
possible to estimate them by comparing different local labor markets with and \^ ithout PVE 
programs. 

Gross placement rates arc also inadequate for calculating PVE's effect on employment of 
under-represented groups in particular occupations. Again, some or all of the jobs obtained by 
target group membcre who participated in PVE would have been filled by other members of the 
same groups if the PVE program did not exist The net placement of under-represented group 
members would be computed by subtracting the number of target group members who are 
displaced from the number who arc placed. Again, net placement rates might be estimated by 
comparing local labor markets that do and do not have PVE programs. 

Data on earnings and non-pecuniary rewards of PVE graduates are no more conclusive than 
gross placement rates. PVE graduates may obtain jobs with high pay and attractive non-monetary 
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characteristics, but some or all of those jobs would have been filled by other job-seckcn if the 
PVE program did not exist Those other job-seckcrs may therefore have to settle for jobs with 
lower wages and less desirable characteristics. Again, some measure of net gain in wages or 
non-pecuniary characteristics could be defined, as the diff/,Tence between the gain to PVE 
participants and the loss to non-participants. Actually measuring the loss to non-participants 
would be very difficult, however. 

The argument here is that opening job opportunities for PVE participants necessarily 
reduces opportunities for non-participants. Counting the gains to participants without counting 
the losses to non-participants therefore ovf r states the gains from PVE. 

This conclusion is just the reverse of the argument put forward by Gustman and Steinmeier 
(1982) about vocational education in high schools. They conclude that vocational education 
might have a positive effect on aggregate earnings even when there is no difference between the 
earnings of vocational and non-vocational high school graduates, if the vocational prc>gram has 
expanded to include all students who want it. In other words, the additional earnings of 
vocational compared to non-vocational graduates would under state the aggregate gains from 
vocational education. This is an important conclusion, bat the model on which it is based 
assumes that labor markets clear in a competitive fashion, through wage adjustment This 
assumption implies, in the Gustman-Stcinmcicr model, that employees who get general training 
on the job must pay for it The empirical accuracy of this proposition is uncertain, given the 
mixed evidence, referred to earlier, on whether wage differences do or do not compensate for 
differences in non-pecuniary characteristics of jobs. Competitive wage adjustment also iniplics, 
in the Gustman-Steinmeier model, that as the number of vocational graduates increases and the 
number of non-vocational graduates decreases, the average wage difference between the two 
becomes smaller. However, in reality the persistence of unemployment and job rationing implies 
that the wage difference between the two groups could actually become larger, as vocational 
graduates who are not needed in high -wage jobs start bumping non-vocational graduates out of 
jobs they would have held, leaving some non- vocational graduates unemployed (Thmow, 1975). 

The discussion so far has explained wny it is difficult to know whether PVE (or any job 
training program) has accomplished its public purposes even if there is clear evidence that the 
program has improved participants' employment and earnings. In fact, however, it is also quite 
difficult to establish whether PVE has really caused participants* employment and earnings to 
improve. The evaluation of PVE and other job training programs is beset by problems of 
selection bias (Ashentcltcr and Card, 1985; BurUess and Onr, 1986; LaLonde, 1936). IdeaUy, an 
evaluation of PVE would accept some applicants to the program at random (assuming there are 
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moi€ applicants than places, which is not always true), and rejected applicants would all receive 
no training or else some weU-dcfmcd alternative treatment. The difference in post-program 
employment and earnings would then be reasonably regarded as the result of the program. 
However, conducting true experiments is always expensive and sometimes impossible. 

In the absence of random assignment^ any differences between PVE participants and non- 
participants could be attributed to unmeasursd, prior differences between the two groups instead 
of to the program. Differential success in the labor market could overstate the true effect cf PVE 
if PVE participants possess greater unmeasured abilities, more productive personality traits, or 
more helpful social connections than non-participants. Conversely, differential outcomes could 
undersute the true effect of PVE if participants had less of these positive, unmeasured 
characteristics. Systematic differences between participants and non-participants could result 
from self-selection into PVE programs, selection by institutions, or both. Given fhc difficulty 
and expense of conducting true experiments, economrtricians have spent considerable effort 
devising statistical procedures to correct for selection bias, and this remain., an active area of 
research. 

Furthermore, actual evaluations of PVE and other employment training programs are 
usually limited to measuring outcomes for only a short time after participation in the program. It 
is very costly to keep following people for decades, and the interest of evaluators and funding 
agencies tends not to last that long. Yet results from the few long-term, longitudinal studies that 
have been done indicate that some effects of schoolwg do not show up in earnings until many 
years later (Bishop, 1985). 

The importance of long-term effects is recognized by vocational educators, according to 
Hoachlander, Choy, and Brown (1988). Vocational educators object to evaluating their programs 
on short-term placement and employers' initial satisfaction, because PVE is rightly intended to 
give St dents a variety of skills that will increase their productivity as their careers advance. 
Chasing job vacancies rnay distract PVE programs from their public purpose of improving 
cmplo) ^es* productivity. The tradeoff between achieving high placement rates and improving 
long-term productivity is more acute if placement is supposed to be "related to training** because, 
if PVE programs focus on giving their graduates an edge in competing for a narrow range of 
jobs, they will presumably emphasize a narrow set of skills at the expense of more generally 
applicable skills and knowledge. More narrow training may not make PVE graduates cs 
productive in the long run even if they do stay in jobs related to their training and, by definition, 
it will not pay off as much as broader training if they t^tkc jobs in other fields. The tradeoff 
between specific and general training is discussed further by Grubb (1 988). 
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Conclusion: Measure FVE Graduates' Productive Capabilities Also 

It is very difficult to get conclusive evidence on how much PVE increases participants* 
employment or earnings. More important, even if it were known with certainty that PVE 
improved participants' success in the labor market, it would not be at all clear whether or how 
much PVE had achieved its public purposes of enlarging the economy's total output of goods 
and services, increasing employees' job satisfaction in the aggregate, or improving access for 
under-represented groups to certain occupations. 

On the other hand, it is unlikely tha* a program achieves these purposes if participants do 
not find employment where they are more productive than they would have been without tho 
program. Positive labor market outcomes for participants may not be sufficient to establish that 
PVE accomplishes aggregate purposes, but they are necessary (unless the Gustman-Steinmcier 
model is true). Therefore, such outcomes should be included as indicators of program success. 

In using data on employment or earnings to measure aggregate outcomes of PVE, there 
should be some attempt to estimate net effects. This means trying to estimate how many non- 
participants in PVE would have been placed in jobs taken by PVE graduates, and what happens 
to the non-participants' employment, earnings, and expressed job satisfaction. In attempting to 
measure how much PVE improves access of under-rcpresentrd groups to certain occupations, it 
would be desirable to find out whether people displaced by PVE graduates are themselves 
members of under-represented groups. Failure to make such corrections seriously reduces the 
value of labor market outcomes as indicators of ^ogram success. 

In addition to labor market outcomes, indicators of the effectiveness of PVE should also 
include measured gains in participants' knowledge and skill. One reason is that labor market 
outcomes are usually measured only in the short run, seldom more than a year after program 
completiOTi, but the increased productivity of PVE participants may take considerably longer than 
that to become evident. However, these long-term productivity benefits cannot be considered 
results of PVE I nless the program actually produces gains in participants' knowledge and skill. 
These gains should be measurable when participants leave the program, even if the payoff in 
productivity and earnings does not occur until much later and is never measured 

Measuring gains in participants' skill and knowledge also verifies whether PVE really 
proiuces aggregate benefits. The problem with relying on labor market outcomes alone is that 
PVE may to some extent serve a pure screening function, enabling participants to obtain more 
employment and higher earnings which are offset by reductions in employment and earnings for 
non-participants. However, if gains in participants' knowledge and skill are measured directly, 
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then these gains provide evidence that the program has augmented society's aggregate productive 
capacity. 

In addition to providing a check on whether PVE increases total output of goods and 
services, measuring increases in students* knowledge and skill also is relevant to the other two 
public purposes of PVE. Adding to the number of under-represented group members who are 
demonstrably qualified for certain occupations has the effect of raising the cost of 
discrimination for employers who avoid hiring them while other employers do. Enlarging the 
pool of qualified members of under-represented groups does not by itself ensure an end to 
discrimination, but it does create the conditio<is which are necessary for affirmative action to 
succeed Likewise, achieving the public purpose of increasing individual satisfaction at work 
(lepcnds on individuals becoTning competent in their chosen fields and possessing the capacity 
for continued growth. These are, to some extent, measurable outcomes of PVE. 

Another reason to measure the gain in participants* knowledge and skill is that this is what 
PVE programs most directly affect Teachers have more control over what students learn than 
over whether they get jobs. This is important if the behavior of tcachm and administrators is to 
become niore closely related to measured perfonnance. 

Although in principle there are good reasons to measure knowledge and skill acquisition as 
outcomes of PVE, in practice it is quite difficult to know what to measure. Three broad 
possibilities are specific occupational competencies, generic cognitive skills, and overall mental 
ability. Specific occupational competencies have been developed by vocational educators and 
employers in recent decades. Their obvious advantage is that they relate directly to knowledge 
and skill required for particular jobs. The disadvantage of specific competencies is that they 
change relatively fast, so that an individual who acquires the right competencies todcy may find 
they have become useless in a few years. Therefore, it is desirable to include some measures of 
skill that do not become obsolete, in recent years psychologists and cognitive scientists have 
made progress in identifying more generic intellectual skills that are involved in learning and 
problem-solving (see Resnick, 1987). However, the measurement procedures that have been 
developed in this research often pertain to limited tasks performed in laboratory settings; their 
validity as measures of general thinking or problem-solving abilities is uncertain. On the other 
hand, more traditional measures of mental ability such as IQ tests or general aptitude tests are not 
designed to measure the distinct intellectual processes studied by contemporary cognitive sriencc. 
Furthermore, ihc power of mental ability test scores to predict actual job performance is a matter 
of current dispute (Schmidt, Hunter, Outerbridge, and Trattner, 1986; Levin, 1988; Sticht, 
1988). 
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There is no simple answer to the question of how to nieasurc the outcomes of PVE. There 
are no short-term measures that are known to correlate highly with individuals' long-term 
productivity. It is true that PVE cannot be considered successful unless it produces positive labor 
maricct outcomes for participants (assuming the Gustman-Stcinmcicr argument docs not hold), 
but there are serious problems in measuring labor market outcomes and in making inferences 
from them, as explained above. PVE also cannot be considered successful if participante do not 
learn anything from it, but exactly which kinds of gains in knowledge and skill to measure is not 
at all obvious. In response to these issues, economists tend to argue in favor of measuring labor 
maiket outcomes, while educators prefer to measure participants' gains in knowledge and skill. 
Benson (1985) has observed that 

"In simplified terms, there are two opposing views about vocational 
education. The dedicated educator takes the position that supply of trained 
people creates its own demand, whereas the economist is more likely to 
hold the opinion that demand for skills will create its own supply— or 
induce such capital-labor or labor-hbor substitutions that any perceived 
shortage of sbUs vanishes. If either position were t?iorougnly correct, 
then no government would need to develop a df-tailed policy for skills 
development** (p. 9) 

A compromise position is to measure labor market outcomes to ensure that PVE is creating 
human capital for which there is some demand, and also to measure gains in participants* 
knowledge and skill to ensure that programs are not regarded as failures merely because such 
demand is temporarily lacking. 

IT. Pursuing Multiple Objectives 

The public purposes of PVE conflict with each other to some extent. Pursuing multiple 
objectives in a finite span of time means there are tradeoffs even if the. objectives do not actually 
conflict These conflicts and tradeoffs are briefly described here. Analytical procedures for 
evaluating programs with multiple objectives arc Jicn discussed. 

Conflicts and Tradeoffs among Objectives of PVE 

Two of PVE's public purposes are increasing total output and improving access of under- 
represented groups to certain occupations. The tradeoff between these two purposes is a 
manifestation of the general conflict between efficiency and equity, which is familiar to 
economists (e.g., Okun, 1975). The equity-efficiency conflict has been analyzed in the specific 
context of student fmancial aid for higher education (Hocnack. 1971), anc n discussions of job- 
training programs this tradeoff is described as the well-known problem of "creaming** (e.g., sec 
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Bamow and Consumine, 1988). If members of under-represented groups are more likely to lack 
certain prerequisite knowledge, or if they are more costly Id teach (for instance, because bilingual 
instructors to teach members of certain linguistic minorities are scarce and therefore expensive), 
then students in a PVE program that recruits members of under-represented groups would be 
expected to achieve lower levels of skill than a program that ignores the equity objective^ 
assuming the two programs have the same amount of resources. Lack of prerequisite skills caii 
be taken into account by using growth in students* skills, in addition to level of skill at the end 
of the program, as a measure of the program*s success. If achieving a given amount of growth is 
more costly with certain groups, however, then simply including growth as an objective does not 
eliminate the conflict between efficiency and equity in PVE. 

On the other hand, it is possible that equity and efficiency complement each other un Jcr 
some circumstances. If some group (e.g., females) has been kept out of a certain occupation just 
because members of that occupation have had a "taste for discrimination," then opening up the 
occupation to the excluded [nt)up makes a fresh pool of talent available. Overall productivity 
would tend to increase. 

In theory, then, increasing economic output and improving access may or may not be 
conflicting objectives. The degree of tradeoff presumably varies from one situation to another. 
Fortunately, no prejudgment about this is necessary. The important thing is to include enough 
information in the performance-monitoring system. The necessary information includes levels of 
skill and knowledge attained, growth in skill and knowledge during the program, and the 
proportions of students belonging to particular unda-rcprescnted groups. 

The remaining purrxjsc of PVE is to facilitate individuals' pursuit of happiness" at work, 
by eriabling them to enter their chv-^scn occupations. Does this contribute to increasing GJ^P, or is 
there a tradeoff between these objectives? Presumably, individuals are both happier and more 
productive in an occupation that they have chosen than one they have been forced into it This 
should be all the more true if individual choice is informed by evidence about what it is like to 
work in various occupations, and hov certain personality characteristics seem to be related to 
success in certain kinds of work. On the other hand, some individuals may be attracted by non- 
pecuniary rewards (e.g., glamor, freedom to express oneself) to occupations where the chance of 
success is so small that many end up unemployed, unproductive, and unsatisfied. Free choice 
includes the option to take a long shot and fail. To fulfill its purpose of supporting free choice, 
PVE should give students objective information about their like ihood of success, but PVE can 
be held accountable only for how much it improves students' capabilities in their chosen fields, 
not *br the wisdom of their choice. 



Free choice for all individuals docs conflict clearly with improving access for certain 
groups. If PVE is to help under-represented groups more, it must help others less. Some 
individuals who do not belong to an under-represented group will therefore have less opportxmity 
to fuL^ill their occupational aspirations. The public purpose of favoring undcr-represcntcd groups 
implies that their satisfaction has higher priority. 

Finally, in addition to the tradeoffs that arise from conflicts among the various purposes of 
PVE, there is a general tradeoff that results from the limits of time. Given the number of 
instructional hours in a program, decisions have to be made About what to teach and what to 
leave out Even if the curriculum is standardized nationally or regionally, local administrators and 
teacheis always have some discretion. Therefore, different programs in the same field might 
develop quite different sets of skills and knowledge. A performance-based evaluation system 
must reckon with this somehow. 

In sum, any PVE program has a multiplicity of objectives, including: 

• positive net labor market outcomes, 

• occupationally specific skills and knowledge, 

• general work-related capabilities, 

• level of achievement of each skill at the end of the program, 

• growth in achievement of each skill during the program, 

• separate accounting of these results for each member of an undcr-reprcscntcd group. 

Furthermore, these objectives should be achieved in a cost-effective fashion. That is, with a 
given budget, a PVE program should be operated so that the only way to achieve more of one 
objective would be to achieve less of some other objective: there should be no "slack." 

Analyzing Cost-Effectiveness of Programs with Multiple Outcomes 

If PVE programs in a given ficJd were responsible for only one objective — e.g., 
maximizing participaiits' score on a single proficiency test— then it would be relatively easy to 
det^-mine the cost-cffectivcnesi of programs at different institutions. Programs that achieved a 
high ratio of mean student score to per student expenditure would be more cost-effective. This 
would be a straightforward measure of "bang for the buck," 
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When multiple outcomes are considered, analyzing cost-effecdveness becomes more 
complicated. For instance, suppose two programs in the same field are to be compared. Both 
enroll the same number of students, but one has a larger proportion of handicapped students and 
the other has a larger proportion of economically disadvantaged. One program achieves bigger 
gains in occupationally specific skills, but the other produces more growii in students' general 
work-related capabilities. To make the comparison as simple as possible, suppose the two 
programs have equal cost Which is more cost-effective? 

One approach to this problem is to combine different outcomes into a single score. If the 
combination procedure is linear, this means assigning fixed weights to the different outcomes, 
giving more weight to outcomes that arc considered more important (e.g., sec Grizzle, 1984). 
The resulting score then can be treated as a single outcome, and cost-effectiveness comparisons 
can proceed on the basis of a single ratio. This is what Levin (1983) calls **cost-utility analysis." 

In practice, assigning weights to different outcomes would require, first, a procedure to 
select judges and, second, a procedure to elicit their judgments about the relative values of 
different outcomes. The first procedure is political. If the weights attached to different outcomes 
are to represent preferences of some constituency, representatives of that constituency have to be 
included. The constituencies interested in PVE include students, teachers, administrators, 
taxpayers, and employers. A legitimate set of weights on different outcomes would have to 
represent these different groups. The judges who assign these weights would have to be selected 
accordingly. 

Once a sample of judges is chosen, there are various possible procedures for determining 
weights. A simple procedure would have each judge assign a number — from one to ten, one to a 
hundred, or whatever — ^to each outci-me, where a larger number signifies greater importance. 
These numbers can then be simpiy averaged across judges to derive weights. 

An example of a more complicated procedure for deriving weights is "conjoint analysis" 
(sec Rao, 1977). This has been used in marketing research to discover what differences between 
competing brands are important to consumers. A sample of respondents is asked to make 
hypothetical choices — in this case, the choice would be among a set of hypothetical programs. 
Each program would be described in terms of outcomes; higu on some, low on others. 
Respondents' ratings of programs can be analyzed to provide numerical weights signifying each 
outcome's importarce. Results would probably be more accurate, in the sense of predicting how 
people might actually choose among programs, than would results from the simpler weighting 
procedure. However, conjoint analysis is usually done with only three or four dimensions to be 
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considered. In PVE it would be difficult to reduce the number of dimensions to less than a 
dozen. 

Whether simple or sophisticated, procedures for eliciting weights are inevitably artificial 
and somewhat arbitrary. Arbitrariness is also a problem in selecting the judges. In practical 
terms, the resulting weights are likely to be unstable. If new weights are generated in subsequent 
years, they are likely to cause changes in the cost-effectiveness rankings of different programs — 
changes that are not due to changes in actual performance. Arbitrary and unstable weiglits will 
not guide a steady effort toward educational improvement, but, instead, will promote fickle and 
superficial changes. 

A fmal drawback of these weighting schemes is that they should be non-linear, but existing 
procedures produce only fixed weights for making linear combinations. Non-linear combinations 
are theoretically preferable because the value of a one-point gain in a particular outcome 
presumably is greater if students are performing at a low level than if they are already at a high 
level C'diminishing marginal utility**). Also, the value of a one-point gain in one outcome may be 
greater if students are performing at high levels on other outcomes than if the other outcomes are 
low ("diminishing marginal rate of substitution"). However^ it is difficult to elicit statements o^ 
preferences that satisfy these theoretical criteria. 

There is an alternative method for analyzing cost-effectiveness which takes multiple 
outcomes into account but does not impose arbitrary, fued weights on different outcomes. This 
is a linear-programming procedure developed by Chames, Cooper, and Rhodes (1978). They 
call the procedure "data envelopment analysis** (DEA). The purpose is to determine which 
"decision-making units,** among a set of similar decision-making units, can be considered 
efficient. PVE programs are an example of decision-making units. The application of DEA in 
education has been demonstrated by Besscnt and Bcssent (1980), who used it to analyze public 
schools in Houston, Texas. 

DEA computes an efficiency index for each decision-making unit The index is the ratio of 
a weighted sum of outputs to a weighted sum of inputs. A different set of weights is computed 
for each decision-making unit, to maximize its efficiency index (constrained not to exceed unity). 
If the maximum efficiency index computed for a given decision-making unit, using the set of 
weights that make it look as efficient as possible, is less than the efficiency index computed for 
another decision-making unit using that same set of weights, then the fu^t decision-making unit 
is deemed relatively inefficient 
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Figure 4 
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Figure 4 Ulustratcs what can be learned from this procedure. A, B, C, D, and E are five 
programs. For simplicity, we consider only two outcomes, and assume here that al! five 
programs cost the same amount of money per student Among the five programs, A, B, and C 
would have maximum efficiency ratings, but D and E would not The lines connecting efficient 
programs represent the production possibilities frontier. Programs A, B, and C are on the 
frontier, but D and E arc not 

Procedures like DEA have certain druwbacks. One problem is that results can sometimes 
change drastically if only one or a few observations are excluded from the analysis. For instance, 
in Figure 4, program E would appear efficient if program C were not included. In addition, 
errors in measurement can produce large distortions in DEA results. Sexton, Silkman, and 
Hogan (1986) discuss these and other drawbacks of DEA, and suggest possible modifications of 
the technique. 

Standard economic procedures for estimating production functions may be more robust 
than DEA against errors in data or model specification, but these standard techniques do not 
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handle more than one output at a time (Forsund, Lovcll, and Schmidt, 1980; Levin, 1974). It is 
possible to handle multiple outputs using a cost function instead of a production function (e.g., 
Cavin and Stafford, 1985), but cost functions can be estimated only if different programs pay 
different prices for the same inputs. The main input to PVE programs is teachers' time, but 
comparing the prices of teachers would require some adjustment for quality, and this is difTicuIt 
Among the available techmques, DEA seems most appropriate for the problem at hand. 

IIL Implementing the Public Purposes of Postsecondary Vocational Education 

PVE may be initiated either by individuals or by employers. PVE programs that respond to 
demand by job seeking individuals take place mainly in schools, where revenues depend on the 
number of students enrolled. Private and proprietary schools depend directly on students' tuition 
payments, and most public schools are reimbursed on the basis of student headcount If students 
do not enroll, schools cannot operate programs. Some individually initiated PVE also occurs 
outside of schools, in firms that offer it to employees on a voluntary basis. Most PVE in 
workplaces, however, is initiated by employers as on-the-job training (OJT), in which 
employees are expected to participate. Although employers may contract with schools or other 
training agetKies to provide OJT, most employer-initiated PVE occurs at the work site (Camevale 
and Goldstein, 1983; Lillard and Tan, 1986). Therefore, the two main agencies that deliver PVE 
are schools, which traditionally specialize in responding to student demand, and firms, which 
mainly provide required OJT. 

Benefit-Cost Consideradons in School-Based and ^irm-Based PVE 

The DEA procedure described above is a method for identifying PVE programs that arc 
relatively cost-effective. However, it is possible that even a relatively cost-effective program 
would not pass a conventional benefit-cost test Cost-effectiveness analysis determines whether a 
program is relatively efficient in converting resources into desired outcomes, wheie neither 
resources nor outcomes have to be measured in monetary terms. A conventional benefit-cost 
analysis, on the other hand, compares the value of benefits and costs, both measured in money. 
In analyzing an educational program, conventional benefit-cost analysis compares the value of 
additional output produced as a result of the program (usually measured by the additional 
earnings of participants) with the program's cost The program is judged to be a worthwhile 
investment if the present value of benefits exceeds the present value of costs when future benefits 
and costs arc discounted at an appropriate interest rate, or if the internal rate of return is higher 
than the rate of return to alternative investments. 
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Since school-based PVE is often subsidized by taxpayei3, students do not pay the fuU cost 
Therefore, it is possible that ^efitt exceed costs for students but not for society as a whole. In 
fact, the subsidization of schooling by government makes the private rate of return higher than 
the social rate of return in all parts of the world and all levels of education, though the social rate 
of return to schooling is still positive virtually everywhere (Psacharopoulos, 1985). The gap 
between social and private rates of return is generally largest at the postsecondary level of 
schooling. Furthermore, the rate of return *o vocational education, both secondary anc^ 
postsecondary, has generally been found to t . ower than the return to academic education, 
according to Psacharopoulos (1987). The reason is that the gain in earnings from additional 
vocational education is about the same as from additional academic education, but vocational 
schooling tends to cost more because it often requires costly equipment and class sizes are often 
smaller. While existing evidence on the social rate of return to school-based PV£ still cannot be 
considered definitive (Breneman and Nelson, 1981; Grubb, 1984), the available evidence does 
not suggest that the rate of return is particularly high. Furthermore, Grubb (1983) has found that 
enrollment growth in school-based PVE programs leading to particular occupations docs not 
conform very closely to growth in those occupations. Students may enroll in subsidized, school- 
based PVE because they are attracted to the non-pecuniary rewards from certain occupations, or 
because they are misinformed about the prospects of employment 

In contrast, employers iniiiate PVE only Jf they expect to profit from it Since employers 
are in a better position than individuals to know what skills and knowledge are in demand, their 
expectations about the payoff firom PVE are likely to be more accurate. Furthermore, only a 
fraction of the gain from PVE accrues to employers (the rest accrues to employees and 
consumers, as discussed below), so the total value of benefits from employer-initiated PVE must 
be expected to exceed its cost by a wide margin. Indeed, there are reasons to believe that 
employers systematically overlook profitable opportunities for OJT. One reason is that employers 
cannot fully insure themselves against the risk of losing employees in whose training they have 
invested (Stem, 1982). Another is that investment in OJT, like investment in physical capital, 
traditionally has had to be justified on grounds of cost saving, rather than improving product 
quality or increasing total sales (Business Week, 1988). Bishop (1985) has argued that imperfect 
capital markets, imperfect information about an individual worker's productive potential, and 
progressive izx rates also lead to underinvestment in OJT. 

For these reasons, economists have proposed subsidizing employer-based training instead 
of, or in addition to, school-based vocational education (Thurow, 1970; Bishop, 1985; 
Psacharopoulos, 1987). Benson (1985) has suggested conditions under which it is more efficient 
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to subsidize OJT instead of school-based training. OJT will be more efficient, according to 
Benson, if the demand for skills is difficult to predict, the technical content of skills is changing 
fast, the pool of potendai trainers is small and there are well-paid alternative uses for their time, 
equipment requirements arc large or changing fast, and trainees arc not familiar with the 
workplaces in which they are seeking employment On the other hand, Benson points out that 
employers are less likely than schools to teach theory, and instruction in theoretica] concepts is 
warranted if jobs in a particular occupation v ary in the complexity of skill required, or if there is 
more opportunity for workers to inviovatc. 

An Example of Subsidized OJT: The California Employment Training Panel and New United 
Motors 

California has a unique provision in its l^nemployment Insurance (UI) program that sets 
aside $55 million each year from the UI fund to support training of individuals who are 
unemployed or "likely to be displaced**. A seven-member Employment Training Panel (ETP) 
allocates money to particular training projects. The authorization to spend money for employees 
who are likely to be displaced means that employers can apply for funds to train existing 
employees in connection with char ges in technology or business operations. In effect, this is 
publicly subsidized OJT. 

An example of how such a program can achieve the public purposes of PVE h provided by 
Ne w United Motors Manufacturing, Inc. (NUMMI). This company is a joint venture between 
General Motors and Toyota. It was formed in 1984 to operate an automobile assembly plant in 
Fremont, California. General Motors had closed the plant in 1982 because it was unproductive 
and unprofitable. Labor relations between General Motors and the United Auto Workers had 
gone from bad to worse. When NUMMI reopened the plant in 1984, however, both the union 
and the new management procl«iimcd their intention to work together in a spirit of cooperation. 
This intention was written into the first collective bargaining contract, which also included 
unusual pledges to protect worker?' employment security. 

NUMMI started with virtually the same plant and equipment General Motors had operated, 
and almost all the employees at NU?VIMI in 1984 were hired from among those who had been 
working in the plant when it closed two years earlier. What was new was the way production 
was organized. The number of job classifications on the factory fioor was reduced from about 80 
to only two. team member and team leader. Most teams consist of four members plus the leader. 
TTie team handles a given section of the line, and has responsibility for dividing the work, 
defining work standards, and maintaining quality. The new production system, along with 
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greater employment security and a more cooperative ethos, have enabled NUMMI to achieve 
approximately 50 percent higher productivity than General Motors did in the same plant (Krafcik, 
1986). 

This phenomenal increase io productivity was facilitated by an intensive program of 
training. The ETP (Employment Training Panel) has invested more than $5 million at NUMMI, 
and the conq>any has also spent a great deal oi^ its own money on training. The first ETP project 
provided training for team leaders and '"group leaders^ (formerly called foremen) in the new 
production system, which was copied from Toyota; 120 hours of that training was conducted in 
Japan. The second ETP project trained team members in setting work standards the Toyota way, 
**kaizen" (which means good change, progress, or continuous improvement), quality, safety, and 
problem- solving techniques. A third ETP project retrained skilled crafts workeis (steamfitters, 
stationary engineers, electricians) to be general maintenance mechanics as at Toyota. A fourth 
ETP project, which started in 1988, is teaching team members about new technologies, as 
NUMMI is now making a reported $200 million investment in retooling its production line. 

One of the interesting aspects of OJT at NUMMI is that participants in classes on problem 
solving, kaizen, and quality can be said to *"do by learning.** For instance, in the problem solving 
class taken by all hourly employees in February and March, 1988, each team picked a real 
problem to analyze. Some of these classroom exercises produced real gains for the company and 
the workers. For example, one team figured out how to eliminate down time in the stamping 
plant This directly increased productivity. It also improved product quality and safety, because 
shutting down the machines and starting them up again damages work in process and produces 
sharp metal pieces which have to be extracted hy hand. Training becomes more affordable when 
it produces such immediate gains. The process of production and the process of learning have 
become closely integrated at NUMMI. 

This example shows how publicly subsidized OJT can serve at least two of the public 
purposes of PVE. Productivity has increased. In addition, there appears to be greater satisfaction 
among employees at NUMMI compared to the period before 1982, as indicated by a great 
reduction in the number of grievances, and enthusiastic statements quoted in newspaper and 
magazine articles. It is not certain whether the ETP-supported OJT at NUMMI has contributed to 
increasing access for under-represented groups, though it is evident from visiting the plant that 
the NUMMI workforce includes substantial proportions of women and racial minorities. 

Since OJT normally is related to the work at hand, there is some danger that subsidizing 
OJT will focus training on very specific skills, yielding only a short-term payoff. However, in 
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Tiq^a,. „ con«m>«. b«efl, fiom PVE the ««,. to it rc^ i, ,ower prices for 
good. «Ki «rvK«. In «ldMon. tMXf^„ benefit fc^ p.id .ddition,l e«nings of PVE 
P«»P«m -K. pddoe «ldilion.Ip„rf|,, orcapW g«n. forowna, . 

.bould be poMible to ,be« effect. forPVE «. whole. to :ev, .«r fee, on PVE 

P«^P«w «KI «». on employe™ tb« would. 00 .vean. 1« t«p,ye„ break even on the 
output-inoeating effect of PVE on me 

Z^^H H T"' *" o^-rrp.^ tro^ to occup^ions 

^ wh>ch the, have previo«.ly been e«l«d.i TO. i. a pnblie r>od in the «ric. economic 
Of « » available to it i. av«lab.e to eveiyone); i,. oKme..,, value i. de,enni«d by 
d« pouncal ptcce.^ »t by n»Ae. e^~ On d» oto ha«I. the «^ 
increased non-moneta,, rew»d, at woA-^k™ only to PVE pmiripmB. not to taxpayers. 

Omdelinaforlnplemaaing Pafbrmemce-BmdPoUcy 

Only a lianted antoum of taxpayer- money i. nrailable for PVE. How ihooU i, be 

a way that ma«ari2e, «:Weveaa.t of aiitegate pyiposes? C3ne approach to this question 
break mnto two pan.: what should be taught, and who should teach it? 

to the question of what shodd be t«.ght presumably varies f™, place to place 
and line to tane. Admimsiruively. there should be some decisioo-making entity -iiat can allocate 
governmental resources for an appropriate geographical area. Tlut agency should maiataii, 
updated estimates of die social «to of reture to education for particular occupaions or 
occupational dusten. li should also keep tuck of whethe dematd i. gro<;™g f„,er than supply 
m each oceupanon. If the mmiber of people employed in a given occupaiion is growing at a fsster 
rete than total employment in the region. a«d the avenge wage in that occupuiou is growing 
faster than the average for an occupMioos. then demand must be growing fa.;,T ih»n ...pply TTie 
regional agency could use this information, doog with rase^f-return estimates, to decide how 
much money riiould be made avulable within its jurisdiction for training in pmicular 
occupations. These allocations should change firorj year to year, but yearly changes should be 
based on multi-year moving averages so that change is not too abrupt 
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Given the aUocation of money for PVE in each occupation or occupational cluster, 
individual ptogrami should compete co At basis of aieasured coit-cffectivencss in achieving 
positive labor market outcomes as wcU ai dcvdopiag participants' kiOT 
DEA, or some better technique if it can be foond, an index of cost^ffectiveness should 
eventually be c<MnpatBd for e^h profianL faeffidem pro^^ 

assistance. If they reinain indfident year after year, Ihey shooid lose their govenmental subsidy. 
Given the itiU-cjiperimenial natmc of DEA, it win take time to make ^ 
The first task is to deirekjp comuioD measures of sltiU md knowledge ditt 
I^^ progrin» in eK± occiq«ional area. It is crudal rim these measTO 
by adfflinistraiors and instmcton m PVE programs. CHherwise they will comply only wirii the 
letter, not riie spirit, of die petfoimance-based system, u put experience wirii perfonnance- 
based education has shown (Gramlich and Kothel, 1975; Stem. 1986; Sps^rii, 1988). The 
process of developing >ooth conqwtcncies'* for JTPA iUustrates the kind of technical work 
combined wirii political consensus-building riut has to occur before a perfonnance-based system 
can be faithfully inqjlemented. 

By separating riie question what to teach from the question who should teach it, the 
ambitious task of implementing perfonnance-based policy is made somewhat more manageable. 
However, one limitation of the guidelines suggested here is riiat they ignore the degree to which 
benefits from PVE accrue to taxpayers/consumers, rariier than to PVE participants or employers. 
It seems appropriate riiat the rate of governmental subsidy should be higher for programs 
yielding bigger benefits to taxpayers. Unfortunately, ae explained above, it does not appear 
I feasible to adjust subsidy rates for differect PVE progr?^ according to riiis criterion. Further 

I inquiry into riie pioportion of benefit fiom PVE riut accrues to taxpayers/consumers would be 

warranted. 

1 
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